

SEE Rate Estimation: 

Model Complexity and Data Requirements 

Ray Ladbury 

NASA/GSFC Radiation Effects 
and Analysis Group 


To be presented by Ray Ladbury at the 2008 Single Event Effects Symposium, Long Beach, CA, 16 April 2008 


SEE Rate Estimates and Confidence 




[Ladbury, NSREC 2007}: Estimating confidence level of SEE rate 
- SEE numbers fluctuate about a mean, m, according to Poisson statistics 


P(jU,n) = 


- m ex : 


p(-/4 


n\ 


a) 


- Expected # of SEE for LETj 

= Fi<Ti = F iC r lim (l - exp(-( (LETi ' LET ")4) s )) (2) 

- Use Likelihood to find best-fit and confidence contours of parameters: 
s lim . LET 0 ,W,s 


L-flp^n,) 


( 3 ) 


i=l 


- Use Figure of Merit to indicate parameters likely to give high rates 

- Highest CREME96 rate for these 10 combinations gives WC rate at CL 

• Unfortunately, CREME96 won’t work for many State of the art parts 

• Can we extend this technique beyond CREME96? 
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Outline 

Where does CREME96 break down? 

What are possible fixes? 

How do we decide on a new model? 

- Do we even need to decide on a single model? 

How do we compare results across models? 
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Breakdowns of CREME96 Model 


deviations 
from RPP 


o C-12 at 1 6 Gbps! 
m rn F-19 at 1 6 Gbps i. 
A Si-28 at 16 Gbps : 
A 0-35 at 1 € Gbps j 
" □ N»-58 at 1.6 Gbps f 
• Br 79 at 1 6 Gbps : 



0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 

LET (MeV x cm 2 / mg) 

• Charge collection volume not RPP 

• a vs. LET curves may not follow a Weibull form (e.g. no saturation) 

• Failure of LET concept and new charge generation mechanisms 
- May change expression for mean and/or add new error sources 
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‘Son of CREME96” Rate calculation 





System 

Requirements 


LET (MeVcrrwmg) 
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IM'ASA 


IM'A^A 


Rate Estimation Fidelity: A Series of Nested Models 

REALITY 

Monte Carlo models: anywhere from 0 to qq parameters 

Improved physics from charge generation to circuit simulation 
Detailed model of device and surroundings from data or fab. 


Figure of Merit: 2-4 parameters 

Approximate environment + 
geometry +charge generation 
and transport 
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Choosing from many models 


Likelihood, L, can only compare modes w/ same complexity 
- More complicated models (> # parameters, k) give better fits 

• Example: Quadratic fit at least as good as a linear fit — even for linear data 


Information Theory approach 

- Hirotsugu Akaike’s Information Criterion (AIC) 

- Asked: Given TRUE model, what is information lost as we move away? 

- Result: AIC = 2(k - In L), k =# of parameters (4) 


- For small datasets use corrected form 

C c = 2(k-lnL) + ^^ + ^^_^_^, n =#of parameters (5) 

Allows comparisons of models w / different complexity 

- Model w/ smallest AlC^most predictive power (favors simpler models) 

- See Akaike, H., “A New Look at the Statistical Model Identification,” IEEE 
Trans. Automatic Control 19 (6): 716-723 (1974) 
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Penalty for Complexity 
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Needed Improvement in L 


How much must Likelihood Improve? 

Remember: AIC c depends on ln(L) 


1.E+10 


1.E+04 


from 4 to 5 
■ from 5 to 6 
from 6 to 7 
-x- from 7 to 8 



5 10 15 20 

# of Data Points 
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Monte Carlo: Developing Models from Data 




Cross Section 


0.0E+00 4.0E-05 8.0E-05 1.2E-04 


Without Manufacturer-supplied model: 
Warren et al. [TNS 8/07, p. 898.] 

- LET 0 determines Qcrit 

- Increase in a gives thickness of new 
sensitive vol. 

- LET/ LET 0 determines charge collection 
efficiency for ith sensitive vol., Sj 

- Fit a vs. LET to Weibull 

Similar to usual CREME approach 

- Still subject to Poisson errors 

- Amenable to likelihood/AIC treatment 

except 

- Cannot assume effective LET 

- # of sensitive volumes may depend on 
when fit saturates and desired accuracy 
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Verifying a Monte Carlo Rate 




• Rate calculated done by Monte 
Carlo simulation w/ CAD model 

• Verification: Look at model 
predictions vs. E, LET, 9... 
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• Monte Carlo predicts p, SEE 
counts for flux, Fj, at LET;, E it 
incident @ angle 0|. Observed=Nj 

- How Significant is disagreement? 

• Poisson((ii,Nj) 

• Other sources of error? 

- MBU, New error modes, etc.? 

• Do we need to modify our model? 

- Do error trends tell us how? 

• Angle? Energy? 

• Frontside vs. Backside? 

• Can also look at multiple models 

- over uncertainties over device 
parameters 

- AlCc selects models that fit data 
most efficiently 
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Model Mj 


ModelMj 

1 Model M 3 1 1 Model M 4 

AIC, 


aic 2 

AICj A1C 4 


Introduce Akaike Weights: W = JT P( 0,5>A1Ci) 

J ^ exp( 0.5*AIC i ) 

1 

Model Averaging: R = J]W i R i Wj measures degree of 

■ support in {OJ for Model Mj 
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Bounding Rates Using AIC 




• SEE rate @ confidence level 
CL defined by highest rate 
over all parameters with 
likelihood within 8 CL of best 
fit parameters. 


• Use AIC to bound rate across 
models as well as across 
parameter values. 



Ex: M 3 =BF, M 2 w/in CL, M,, M 4 outside CL— »R WC =MAX(R 2 ,R 3 ) 
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Conclusions 

• Statistical Methods outlined in [Ladbury, TNS2007] can be 
generalized for Monte Carlo Rate Calculation Methods 

• Two Monte Carlo Approaches 

- Rate based on vendor-supplied (or reverse-engineered) model 

• SEE testing and statistical analysis performed to validate model 

- Rate calculated based on model fit to SEE data 

• Statistical analysis very similar to case for CREME96 

• Information Theory allows simultaneous considerationof multiple 
models with different complexities 

- Model with lowest AIC usually has greatest predictive power 

- Model averaging using AIC weights may give better performance if 
several models have similar good performance 

- Rates can be bounded for a given confidence level over multiple 
models, as well as over the parameter space of a model 
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